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Worst  Case  Analysis  of 

Greedy  Heuristics  for  Integer  Programming  with  Non-negative  Data 


Greg  Dobson 

Systems  Optimisation  Laboratory 
Department  of  Operations  Research 
Stanford  University 
Stanford,  California  94305 


Abstract 

We  give  a  worst  case  analysis  for  two  greedy  heuristics  for  the  integer 
programming  problem  minimise  cz,  Az  >  b,  0  <  z  <  u,  x  integer,  where 
the  entries  in  A,  b,  and  c  are  all  non- negative.  The  first  heuristic  is  for  the  case 
where  the  entries  in  A  and  b  are  integral,  the  second  only  assumes  the  rows  are 
scaled  so  that  the  smallest  nonsero  entry  is  at  least  1.  In  both  cases  we  compare 
the  ratio  of  the  value  of  the  greedy  solution  to  that  of  the  integer  optimal.  The 
error  bound  grows  logarithmically  in  the  maximum  column  sum  of  A  for  both 
heuristics. 
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Consider  the  integer  programming  problem 

minimize  cx 

Ax  >  b  (P) 

0  <  i  <  u  x  integer 

with  the  additional  restriction  on  the  data  that  it  be  nonnegative  (i.e.  a,y  >  0, 
bi  >  0,  Cj  >  0  for  all  i,j).  We  establish  tight  bounds  on  the  worst  case  behavior 
of  two  greedy  heuristics.  If  the  constraint  data  is  either  integral  or  if  each  row 
is  scaled  so  that  the  smallest  entry  is  at  least  1,  then  the  error  bound  tends  to 
grow  logarithmically  in  the  maximum  column  sum  in  “A”.  The  results  here  are 
a  direct  generalization  of  the  work  of  Chv4tal[l]  on  the  set  covering  problem  and 
of  Lov4sz[4]  and  Johnson[3]  on  the  unit  cost  set  covering  problem. 

In  §  2  we  describe  the  basic  algorithm  and  give  a  proof  for  the  case  of 
integral  constraint  data  and  no  upper  bounds.  In  §  3  the  analysis  is  extended 
to  the  case  where  upper  bounds  are  allowed.  §  4  deals  with  the  case  of  non- 
integral  constraint  data.  Here  it  is  necessary  to  introduce  a  modification  of  our 
basic  algorithm  to  maintain  a  logarithmic  error  bound.  Finally  §  5  gives  some 
evidence  why  the  non-negativity  of  the  data  cannot  be  dropped  and  still  have 
an  approximation  algorithm  that  runs  in  polynomial  time.  This  section  also 
shows  why  certain  improvements  in  “greedy”  heuristics  cannot  provide  better 
than  logarithmic  error  bounds. 


2.  The  Basic  Algorithm  and  Error  Analysis 

Throughout  this  section  we  assume  that  the  constraint  data  is  integral  and  that 
there  are  no  upper  bounds  on  the  variables. 

The  greedy  heuristic  picks  the  column  /  that  minimizes  cy/£”Lj  <*ij,  incre¬ 
ments  x±  by  1  and  repeats  the  process.  Column  j*  minimizes  the  myopic  unit 

cost  of  satitfying  the  constraints.  Observe  that  when  we  have  o,j  >  6,  then 
setting  Xj  —  1  would  satisfy  the  t’th  constraint  yet  this  “large  a,y”  value  could 
make  the  ratio  Cj/YliLi  a*j  appear  artificially  small.  In  order  to  obtain  any 
bound  at  all  it  will  be  necessary  to  adjust  the  matrix  data  during  the  algorithm 
to  elliminate  “large  Of/s” .  In  particular  at  the  end  of  every  iteration  we  adjust 
any  a#  >  b,-  down  to  i.e.  a#  <-  min(a,-;-,  6,).  For  simplicity  we  assume  this 
adjustment  has  been  made  to  the  original  data  so  o,y  <  b,. 

The  greedy  heuristic  is 
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Greedy  1  (no  upper  bounds) 
x  i —  0 
z  0 

while  6  7^  0  do 
begin 

k  <-  argmin  i<j<*(e,7 EZLi  <*<i) 

Zk  ♦“**+! 

6*  «—  6*  —  <*<*  for  * 

o,j  4-  minfoy,  6,)  for  all  s',  j 
z  +-  Z  +  Ck 
end 

Denote  by  H(d)  the  first  d  terms  of  the  harmonic  series:  H(d)  =  £)f=1  V*- 
We  can  make  the  following  performance  guarantee  on  the  value  of  the  heruristic 
solution: 

Theorem  2.1.  Given  problem  (P)  with  integral  constraint  data  and  no  upper 
bounds  on  the  variables,  if  x  is  the  optimal  integral  solution  and  2  is  the  solution 
given  by  the  greedy  heuristic  then 


c2 


cx 


<  H 


and  this  bound  is  tight. 

To  see  that  the  bound  is  tight  consider  the  program  : 
minimize  \x\  -f  3^1*2  +  •  *  *  +  1  +  f *d  +  (1  +  0X*+ 1 


z\ 

+ 

Zd+ 1  >  1 

Z 2 

+ 

Zd+1  >  1 

Zd-1  + 

Zd+l  >  1 

Zd  + 

Zd+1  >  1 

where  Zj  =  0, 1.  The  heuristic  picks  the  solution  2  =  (1, . . . ,  1,0)  whereas  the 
optimal  solution  is  z*  =  (0, . . . ,  0, 1)  for  every  e  >  0.  In  this  case  as  c  -►  0 

=  rr-  - 

cz*  1  + c 

Because  the  data  (A,  4)  is  changing  throughout  the  algorithm,  we  introduce 
the  notation  Ar  =  (a[;)  and  bT  =  (6[)  to  refer  to  the  data  at  the  start  of  the 
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r’th  iteration.  Let  w*-  =  Y^TLi  ai3-,  that  is  the  j'th  column  sum  at  iteration  r. 

Assume  that  the  algorithm  terminates  after  t  iterations,  that  is  6t+1  =  0.  At 
each  iteration  r,  it  picks  column  kr,  z*f  is  increased  by  1,  so  “6,”  is  decreased 
by  Thus  i’+l  =  6;  -  «;*,  ot = »;  - i;+1. 

Note  that  if  column  kT  is  picked,  it  is  picked  at  least  p  =  ^min  [6J/a[fcJ 

times  consecutively.  To  see  this  observe  that  the  p’th  time  -will  be  the  first  time 
that  one  of  the  o,*r’ s  will  be  reduced,  thus  the  first  time  a,-*r  will  be 

reduced.  Even  though  the  other  column  sums  are  (possibly)  being  reduced  by 
the  6,-’s,  that  only  makes  those  columns  less  attractive.  Therefore  any  good  im¬ 
plementation  would  always  increment  z*r  by  p.  Suppose  we  implement  the  algo¬ 
rithm  this  way.  Once  z*,  is  choosen  where  t  =  argmin  |X/a'fc rJ, 

'  1  < » <  m 

thus  if  z*r  is  choosen  again  then  row  t  is  covered.  Only  m  variables  could  be 
choosen  twice  and  we  have  that  the  number  of  iterations  is  bounded  by  n  -f-  m; 
thus  the  heuristic  is  polynomial.  For  ease  of  notation  we  will  use  the  former 
description  of  the  heuristic. 

To  prove  Theorem  2.1  we  will  need  some  machinery  provided  by  the  lemmas 
below.  In  particular  we  introduce  two  sets  of  “price”  functions  which  will  be 
useful  in  comparing  the  value  of  the  heuristic  solution  and  all  other  feasible 
solutions. 

Thus  introduce  a  set  of  step  functions  p,(«)  that  will  represent  prices  paid 
per  unit  by  the  greedy  heuristic  to  satisfy  the  constraints.  We  wish  to  view 
satisfying  the  t’th  constraint  (££Li  a*jxj  >  &«)  38  covering  up  the  interval  [0,  i>,). 
In  particular  we  will  cover  it  up  from  right  to  left  so  that  the  interval  remaining 
to  be  covered  at  iteration  r  is  [0, 6$).  Intuitively,  for  each  point  s  in  the  interval 
[0,  bi)  the  unit  price  paid  by  the  greedy  to  cover  this  point  is 


PM  = 


if  for  r  =  1, . . . ,t 


That  it  is  indeed  the  exact  price  will  now  be  shown. 


Lemma  2.2. 


el  = 


Pi{s)ds. 


Proof. 
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We  now  define  a  unit  price  function  Pij{s)  for  each  element  of  the  matrix 
analogous  to  the  way  we  defined  p,(s)  for  each  6,.  Define 

if  «  £  K^.ojy)  for  r  = 
if  s  £  [a}3-,  6,) 

We  hare  that  p, •_,(«)  is  non- increasing  for  s  in  [0, 6,)  since  to'-  is  non-increasing  in 
r.  Intuitively  p„(s)  is  the  price  that  would  be  paid  to  cover  the  point  s  in  [0,6,) 
using  column  j.  Because  the  heuristic  is  myopic  we  have 

Lemma  2.3.  If  s  £  [0, 6,)  then  p<(s)  <  Pij(»)  for  all  j. 

Proof.  Fix  j.  Let  r  be  the  iteration  number  such  that  s  £  [a^1,  ojy).  Because 
“a,y”  was  reduced  at  iteration  r,  we  must  have  ojT1  =  6JJ+1,  and  clearly  Oy  < 
b\,  hence  s  £  [6,r+1,6J). 


wW  =  ^-<^  =  PiiW 


where  the  inequality  follows  from  the  choice  rule.  | 

Theorem  2.4.  Let  /:[0,6)  -+  (0,oo)  be  non-increasing,  a  £  (0,6],  S  C  [0,6), 
H (5)  >  a,  then 


W)fsf* 


1 

a 


Proof.  Let  A  =  [0,  a),  then 


i  1  [  f)+f^zA>(  i  [  f) 

as)Jsj  ms)  \i‘isnA)Jsn*)  ms)  W-a)/s_ a1) 
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This  is  a  convex  combination  of  two  averages.  Since  /  is  non-increasing  the  first 
average  is  at  least  as  large  as  the  second.  Thus  a  convex  combination  of  these 
averages  weighted  more  heavily  on  the  first  term  can  only  be  larger. 


< 


MgjW  i  f  A  ,  MA-s)(  i  [  • 

ma)  VMsnA)/snA ;  ma)  Vrts-^Js-A1. 


Since  /  is  non-increasing,  f  on  A  —  5  is  at  least  as  large  as  /  on  S  —  A,  thus 
Ss—a  f  —  S)  /a—  s  /•  After  making  this  replacement  in  the  second 
term  we  have 


< 


MsfW  i  [  ,  KA-S)(  i  f 

H{A)  \fi(AC\S)JAQsJ)^  p(A)  (f t(A-S)JA-s\ 

JLff. 

u(A)JaJ 


I 


We  can  now  easily  prove 


Lemma  2.5. 


jj“  /  Pi{s)  ds  <  f  Pij{s)d8. 

•'(O.Si)  ^[0,ay) 


Proof.  First,  since  p,(s)  <  p,j(s)  for  e  £  [0, 6,),  by  lemma  2.3,  we  have 

—  pi(s)  ds  <  i  f  pij[s)  ds. 

Second,  p^  is  non-increasing  in  s  since  to'  is  non-increasing  in  r.  Apply  Theorem 
2.4  with  S  =  [0,  bi )  and  /  =  pij  to  obtain 

r-  f  Pij(«)  ds  <  —  f  pij(s)  ds. 

bi  J[ 0,bt )  Ofj 

Combining  the  last  two  inequalities  the  result  is  immediate.  | 

For  the  set  covering  problem  where  a*  =  0  or  1  and  6,  =  1  for  all  t,  j  a  row 
t  is  either  not  covered,  (bT{  =  1),  or  completely  covered,  (&J  =  0),  thus  the  price 
function  Pij(s)  only  takes  on  one  value,  X,-  =  c*r/to£r  where  row  t  is  covered  at 
iteration  r.  In  Chv4tal’s  analysis  of  the  greedy  algorithm  for  the  set  covering 
problem  he  proved  the  inequality 

™  /A  \ 

22  X‘°‘j  ^  e3H  I  zJav)  foT  J  =  !>■■■,  *• 

These  are  relaxed  dual  constraints  of  the  associated  linear  programming  problem. 
We  are  now  in  a  position  to  prove  the  analogous  inequality  for  this  problem.  For 
each  j  define  Vj  =  min{r  |  =  0}. 
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Lemma  2.6. 


where 


m  /• 

52  ~r  LkxPi^da  -  Cjhj- 


Vj  V)r.  —  Wr-~^ 1 

Y'  3  3 

2-4  ./,r 


r  =  1  3 


Proof. 


53  IT  f,  x  da  -  52  f,  ,  Pi3  M ds 

ft.  •'[0,60  yt°.“«) 


m  v*  c  ■ 

=  EE>:r<ri 

t=i  r=i  j 

=  c>  £  -  «’5+1) 

r—  1  J 

==  Cjh3-.  | 

With  Lemmas  2.2  and  2.6  we  now  complete  the  proof  of  Theorem  2.1.  By 
Lemma  2.2 


— SL"«* 


Now  let  x  be  any  feasible  solution,  so  2_  o»j*j  >6,  or  2^  — r —  >  1,  thus 

j=i  i=i  ‘ 


sS(S^)L*«* 

-s(s¥L-<->4' 


<  Ew*i 

i=i 


By  lemma  2.6  we  have 


§3 
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Cl 

—  <  max  h-i 
cx  1  <3<n 

for  all  feasible  z.  We  now  use  the  assumption  that  the  “o,y’s"  and  “6,’s”  are 
integral. 

h<  =  y  — — i—  =  v  v  i. 

"  As  As  yjf 

r—1  3  r=l  w'i+1<l<v>'  3 

i  i 

sE  E  7=  E  7 

r  —  l  w'+i<l<v>' 

= «(*})  = »  (e  ««) 

max  A,-  <  max  iff  Y"*  a,-,  )  =  HI  max  Y  a,,- ) 

i<j<"  i<j<»  V“J  J  / 


Thus 


Therefore,  since  the  optimal  solution,  z,  is  feasible  the  final  bound  is 


|  <  ii  <  »(  m« 

cz  cx*  \i<3<»tr[  ) 


S.  The  Analysis  with  Upper  Bounds 

We  now  extend  the  previous  result  to  the  integer  programming  problem  with 
upper  bounds. 

minimize  cx 

Ax  >  b  (P) 

0  <  z  <  u  x  integer 

Again  we  assume  atj,6„cy  >  0  for  all  s’,  j.  If  u  =  (1, . . . ,  1)  we  have  the  important 
special  case  of  0-1  variables.  The  primary  difference  in  the  algorithm  is  in  finding 
that  Xj  has  reached  its  upper  bound  Uj,  the  column  Aj  is  set  to  0  to  prevent  the 
column  from  being  picked  again.  Because  *Oy”  does  not  decrease  with  “&,•*,  i.e. 
oJ;-  =  min(o,-,-,6J)  does  not  always  hold,  lemmas  2.3  and  2.5  break  down. 
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Greedy  1  (-with  upper  bounds) 
x  «-  0 
z  <—  0 

while  6  0  do 

begin 

k  <-  argmin  i <j<n(cj/ £,wi  «y) 

Z*  «-  z*  +  1 

6t-  4—  bi  —  a,*  for  all  t 

Oy  min(aij,  6,)  for  all  i,j 

z  4 -  z-f-Ck 

if  xk  =  u*  then  A*  4—  0 

end 

The  extension  of  Theorem  2.1  is 

Theorem  3.1.  Given  problem  (P)  with  integral  constraint  data  and  with  optimal 
solution  x,  if  2  is  the  solution  given  by  the  greedy  algorithm,  then 


<  H 


Proof.  For  the  moment  we  restrict  our  attention  to  constraint  i,  and  let  x  be 
a  vector  that  satisfies  that  constraint,  (22j=i  avxJ  >  &«)•  The  difficulty  arises 
for  those  variables  2y  which  reach  their  upper  bounds  uy  before  the  algorithm 

stops.  Define  i,  =  min{r  |  =  0}.  Define  [/,■  =  {j  j  2y  =  tiy  but  ktt  ^  j), 

i.e.  those  variables  that  reached  their  upper  bounds  before  the  iteration  that 
covered  row  For  j  E  Uit  define  W,y  —  {iterations  r  \  kr  —  j}.  W,y  is  the  set 
of  iterations  where  column  j  is  picked,  and  2y  is  incremented. 

Say  Wij  =  {sx,  ...,sUf}.  Using  the  feasible  x  we  define  Vyy  =  {si, . . . ,  sXj}, 
i.e.  the  first  uXj”  values  from  W,y.  We  now  divide  the  interval  [0, i>,)  into  two 
parts.  On  each  part  we  compute  a  bound  for  the  price  paid  by  the  greedy 
heuristic  to  cover  that  part.  Define 

Ri  =  U  f6i+1-  *0  s<  =  b< )  ~  R<- 

'€Vy 

J€Vt 


Claim: 


/  Pi{*) 

JPi 

min(o,y,  fi{Si))  f  , 

—m  krM 


ds 


ds 


(3.1) 

(3.2) 
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Proof  of  (3.1).  First  observe  that  for  j  6  Ui,  “ay”  only  takes  on  the  values  0  and 
ojy.  To  see  this  assume  ^  ari}-  =  bTi  ^  0,  that  is  ay  took  on  another  nonzero 
value.  At  this  iteration  23  <  u3  (otherwise  ar^  =  0).  Hence  j  €  U<  implies 
column  j  is  picked  again  and  covers  row  t,  kti  =  j,  thus  j  £  Ui  contradiction. 


P«(s)  ds 


Now  apply  that  fact  that  ay  only  takes  on  the  values  0  and 


Notice  that  maxr€wry  is  the  largest  value  of  c3/wy  before  column  j  is  set 

to  zero  and  u>'-+1  =  0,  and  thus  is  the  one  that  defines  pij  on  [0,  ay). 


Proof  of  (3.2).  Note  5,-  ^  0  since  [0, 6*()  C  S,-.  Define  ay  =  min (ay,/i(S,)). 
Since  j  £  U{  lemma  2.3  holds,  so 


by  Theorem  2.4 


Ta~\  /  PiWds  <  “tVt  /  P<iWdS 

p(Si)  Jst  n{Si )  JSi 

xPy(s)ds 

fly  j[ 0,a«y) 

<  ~  [  Py00d«- 

aij  >'[0,ay) 


I 


We  complete  the  proof  of  Theorem  3.1.  Y,j£ut  a*ix3  ^  ~  Xyec/,  0yzJ  = 
6,-  —  M(fi«)  =  A»(5*)-  Recall  that  ay  =  min(ay,/i(St)),  so  8<#*i  ^  M5*)- 
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By  Lemma  2.2  we  have 


m  f  m  i» 

=  X)  /  PMd8  +  Yl  Pi(*)d* 

«•=  l*'s< 

If  i  is  feasible  °<jx3  —  /*(^«)-  thus 

/.«•>* 


Applying  (3.1)  and  (3.2) 


-  £ + £ £ {Lj<Mi,)zs 
=s£Wo.*«»pi,w4j 

=  E(EE~K-C')>, 

j=1  \=1  r=l  i  7 

i=i\=i  i  7 


The  rest  follows  as  in  the  proof  of  theorem  2.1. 

m 


(m  v  n 


Finally  if  z  is  the  optimal  solution, 


e2 

cz 


<  —  <  iff  max  V]  a*jl 

-  C2*  "  Vi<y<«,t1  v 


4.  Cate  of  Non-integral  Constraint  Data 

We  now  wish  to  drop  the  restriction  on  the  integrality  of  the  constraint  data,  A 
and  6.  Because  we  are  now  free  to  scale  the  rows  so  as  to  make  a  column  sum 
as  small  as  we  like  we  need  a  standard  form.  We  thus  assume  that  each  row 
(an,. . . ,  atn,  bi)  has  been  scaled  so  that  the  smallest  non-sero  entry  is  at  least 
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1.  Unfortunately  the  error  of  the  greedy  heuristic  can  be  as  bad  as  linear  in  the 
number  of  rows  as  the  following  example  will  demonstrate. 


min  4*1  +  5*2+113+  |l4  +  XB  +  Xg  +  XT  +  Xg  +  P*9 

xl  +  X5  +  (1  •+■  ^)x9  ^  (1  +  c) 

x2  +  Xg  +(1  +  )x9  >  (1  +  C2) 

X3  ~\~x7  +(1  +  ^)x9  ^  (1  +  €3) 

X4  +  Xg  +  (1  +  64)*b  >  (1  +  C4) 

where  i,-  =  0, 1  for  s'  =  1, . . . ,  9  and  f)  =  1  -f-  £  +  c2  -f  e3  +  *4  +  c8- 

The  heuristic  picks  X\  —  1,  xa  =  1,  *3  =  1,  X4  =  1  in  that  order  at  which 
point  the  problem  is  reduced  to 

minimize  z5  ®g  -f-  17  +  Xg  -+-  fix e 

CXg  +  CXg  >  c 

C2x6  -j-  62®9  >  £2 

c3®7  +  C3Xg  >  €3 

64 Xg  +  C4®9  >  e4 

Now  the  heuristic  picks  Xg  =  1,  ®g  =  1,  X7  =  1,  x8  =  1,  thus 

2  —  (1,1, 1,1, 1,1, 1,1,0) 

whereas  you  may  verify  that 

x*  =  (0,0, 0,0, 0,0, 0,0,1) 

so  the  error  is 

«  ff(4)  +  4. 
ex 

The  extension  to  an  arbitrary  number  of  rows  is  obvious.  In  general  the  error 
can  be  shown  to  be 


““»Hl;B'i)+1+<'y} 

where  dj  is  the  number  of  non-zero  entries  in  column  3.  But  by  a  slight  modi¬ 
fication  of  the  algorithm  we  can  replace  this  dj  by  H{dj)  and  regain  a  logarithmic 
error  bound.  The  basic  idea  of  how  this  is  done  can  be  seen  in  the  last  example. 
The  main  component  of  the  error  in  the  heuristic  solution  did  not  enter  until  an 
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element  in  a  row  (  and  thus  all  the  elements  in  that  row)  was  below  1.  Once  this 
occurs  in  all  the  remaining  rows  we  can  scale  the  reduced  problem  so  it  looks  like 
a  set  covering  problem.  At  this  point  Chv&tal’s  original  analysis  (or  Theorem 
2.1)  tells  us  the  error  is  at  most  H(maxi<j<n  dj).  Let  £  be  a  fixed,  arbitrarily 
small  number. 

The  new  algorithm  is, 


Greedy  2: 
x  < —  0 
z  *—  0 

while  6  ^  0  do 
begin 

for  each  j,  if  Y^Li  a*j  <  1  then 

for  any  row  i  such  that  o«y  ^  0  has  not  been  scaled  so  far, 
scale  row  t  to  6. 

k  +-  argmin  1  < y<  n(  ey/  ££  t  o,y) 

Xk  Xk  +  1 

b{  +-  bi  —  aik  for  all  i 
a,j  min(atvj-,  6.)  for  all  i,j 

Z  4-  Z~j~Ck 

if  Xk  =  uj i  then  A*  0 
end 

The  main  result  is 

Theorem  4.1.  Consider  the  program 


minimize  cx 

Ax  >  b  (P) 

0  <  *  S  «  *  integer 

where  a^,  6„  cj  >  0,  each  row  has  been  scaled  so  that  the  minimum  non-zero 
entry  is  at  least  1,  and  dj  is  the  number  of  non- zero  entries  in  the  j’th  column  of 

A.  If  x*  is  the  optimal  solution  and  Greedy  2  is  applied  to  (P)  giving  a  solution 
i  then 

~  <  ,  max  (log( £  aij)  +  1  -f  H{dj) 
cz*  l^n  l  ) 


Proof.  Fix  j.  Recall  from  the  proof  of  Theorem  3.1  that  the  only  step  that  used 
the  hypothesis  of  integrality  of  the  constraint  data  was  in  showing  that 


1 


w)  -  wT.+i 


V) 


r 

3 


L 


Case  of  Non- integral  Constraint  Data 


Thus  to  prove  Theorem  4.1  it  suffices  to  show  that 

V4  tnr.  —  /  m  \ 


vy  —  tt>  . 1  *  /  m  \ 

E  - <  >og(E  + 1 + »(<*/)■ 


Let  g  be  the  iteration  such  that  w?  >  1  but  to?-*-1  <  1.  We  claim 


'.  —  •nf+1 


1  t0  «-  —  to .  1  /  m  \ 

S^s,<S-*)+‘ 


fy  tOf-  —  tOr.+  1 

V  _i - i — 

toj 

r=9+l  3 


<  H{dj). 


The  scaling,  however,  does  cause  one  problem  with  the  previous  analysis.  At 
some  iteration  r,  otJ-  may  be  scaled  down  to  6.  Since  pij  and  p<  reflect  the  prices 
paid  by  the  greedy  heuristic  and  the  heuristic  does  not  actually  cover  up  the 
interval  [tf ,  a^-)  we  should  not  include  this  cost.  To  avoid  this  difficulty,  we  define 
ajy  to  be  the  value  of  atJ-  at  iteration  r  before  any  scaling  and  a^-  the  value  after 
scaling,  and  similarly  for  ©J-  and  to'-.  If  we  no  longer  integrate  over  the  intervals 
[oy,a$y),  the  analysis  goes  through  and  we  have 

d  ^  a  -  ®;+1 

—  <  max  >  - . 

CX*  ~~  l^nr=l 

Since  to'.+1  <  ffi'.+1  we  may  replace  ©'-+1  by  to'+1  and  the  bound  is  only 
larger. 

To  see  why  (4.2)  is  true  observe  that  after  iteration  q,  each  non-zero  entry 
in  column  j  is  equal  to  6,  and  thus  t o^1  <  djS.  Furthermore  since  6  <  1,  if 

7^  0  then  every  nonzero  entry  in  row  t  is  the  same,  6.  Thus  if  a,-y  is  reduced 
again  it  is  reduced  to  0.  The  column  sum  must  be  reduced  by  some  multiple  of 
6.  Upon  factoring  out  the  6  from  top  and  bottom  of  “  (©J-  —  ”,  we 

see  that  the  sum 


»y  ttfl- 

E  - 

r=?+l 


may  be  analyzed  as  before  in  the  case  of  integral  data;  wq.JrX / 6  <  dj,  thus 


t 

1 


wr 

3  3 


<  H(dj ). 
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To  obtain  the  bound  (4.1)  we  solve  the  optimisation  problem 


Za  — 


q  wr.  —  tuf.+1 

V''  3  3 

maximum  >  _ - - - 


f=l 


*5 


s/t  w)  >  •  •  •  >  «t>J  >  1  u>J+1  >  0 

where  w}-  is  viewed  as  fixed,  the  rest  as  variable.  This  is  solved  in  Lemma  4.2 
and  zq  is  shown  to  be  converging  up  to  log(wj)  -f- 1  as  q  — ►  oo  in  Lemma  4.3. 

Lemma  4.2. 

Proof.  For  ease  of  notation  define  B  =  wj  and  make  the  substitution  yq+  i—r  = 
id'-  for  r—  l,...,qr-f- 1. 


zq  =  maximum  ^ 

r=l 


Vr  ~  Vr~\ 
Vr 


s/t  B  >  yq  >  •  >  yx  >  1,  f/o  >  0. 


This  is  equivalent  to 


Vr-l 


zq  —  q  —  minimum  ; 

rtl  Vr 

s/t  B  >  yq  >  •  •  •  >  >  1,  y0  >  0. 

Observe  that  any  optimal  solution  will  have  y0  =  0,  yq  =  B.  We  can  now  solve 
the  minimization  problem  via  a  dynamic  programmming  recursion.  Define 


V(B,q)  =  minimum  ^ 


Vr-1 


r=l  Vr 

s/t  B  >  yq  >  •  •  •  >  yi  >  1,  y0  >  0. 

By  the  comments  above  V(B,  1)  =  0  and  we  now  show  by  induction  on  q  that 

=  (4.3) 


- tr  . 
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Assuming  (4.3)  holds  for  q, . . . ,  1  we  compute 

V(B,q-\- 1)  =  minimum  -f  (q  — 

fl>y>l  \o  J 

Taking  the  first  derivative  with  respect  to  y  and  setting  the  result  to  0  we  have 


Solving, 


&  =  s^-O/v. 


If  B  >  1  then  B  >  $  >  1  and  upon  taking  the  second  derivative  we  see  it  is 
always  positive  for  y  >  0,  so  £  is  a  global  minimum; 


7— y-VU-O-a)  >  o  for  y  >  0 
[q—1) 


Substituting  back  we  have 


VXB,<7+  1)  =  B<'  +  (?- 


B 

=  qB~1 /«. 


Thus 


*«  =  g— (<7— 1)(«})  1/(?  1)- 


Lemma  4.3. 


/(<?)  =  1  —  1  =  <7(1  —  B  */«)  t  log(B)  as  q  ->  oo. 


Proof. 


/'(<?) 

=  (l  -  B"1/*)  +  ^-I/q^jB"1^  log (B) 

-  1  -  B“l/<^1  +  l0g^B^  >  0 

iff 

>(i +'*<*>) 

V  <7  / 

iff 

ilog(B)>log^l  +  ilog(B)j 
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iff  a  >  log(l  +  a) 

with  a  =  J  log(B)  so  f  >  0  always. 


lim  /(<?) 
$-♦00 


by  l’Hopital’s  rule 


lim 

q-+oo 


lim 

q—+oo 


1-B~ */« 
l/<7 


-(l/^B-^logJB) 


— l/qf2 


=  log(B). 


Since  f'(q)  >  0  for  all  q  and  the  limit  is  log(B)  we  have  then  that  zq  converges 
monotonically  up  to  1  +  log(B),  as  q  — ►  oo.  | 


5.  Some  Negative  Results 

A  much  more  satisfactory  result  for  an  approximation  algorithm  would  be  one 
that  gave  a  fixed  bound,  r,  on  the  error  independent  of  the  size  of  the  problem 
data.  If  no  such  algorithm  existed  one  might  hope  to  at  least  show  that  the 
existence  of  such  an  algorithm  implied  P  =  M  P  (see  [2]).  Alternatively  a  nice 
extension  of  the  above  results  would  be  to  the  case  where  negative  numbers  were 
allowed  in  the  matrix.  The  following  result  suggests  that  such  an  extension  is 
unlikely.  This  technique  is  analogous  to  a  result  of  Garey  and  Johnson[2]  on  the 
maximum  independent  set  problem. 

Consider  the  integer  programming  problem 

z\  —  minimum  cx 

Ax  >  b  (Pi) 

Xj  =  0, 1  j  1, . . . ,  ft 

where  «<y  6  {0, 1,  — 1>,  bt  6  {0, 1>,  e3-  E  {0, 1}  for  all  ij. 

Theorem  5.1.  Let  £  be  the  optimal  solution  to  (Pi)  and  let  I  be  a  solution  given 
by  am  approximation  algorithm.  Either  there  is  a  polynomial  approximation 
scheme  to  solve  (Pi)  such  that 

—  <  1  -f  c  for  arbitrary  e  >  0 
ex 

or  there  does  not  exist  any  polynomial  approximation  scheme  that  gives 

—  <  r  for  any  r  >  1. 
cx* 


First  we  need 
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Lemma  5.2.  Given  any  problem  of  the  form  Pi  we  can  create  another  problem, 
call  it  Pa,  of  the  same  form  with  optimal  value  z*  such  that 

*»  = 

Furthermore,  given  anv  solution  to  P*  with  value  w*  we  can  extract  from  it  a 
solution  to  problem  Pi  with  value  v\  such  that 

<  (*>*)*• 


Proof.  For  simplicity  we  start  with  the  case  k  =  2.  Consider 

n 

Z2  =  minimum  53(c,c)x‘' 

*w  1 

s/t  Ax *  -  by{  >  0 
Ay  >  b 

*$  =  0, 1  p,  =  0, 1 


where  c,c  is  the  vector  c  times  the  scalar  c,*,  xx  is  an  vector  for  »  =  1, . . . ,  n,  y 
is  an  n- vector,  6y,  is  the  vector  b  times  the  scalar  y,-. 

It  is  easy  to  check  that  P2  is  a  problem  of  the  form  Pi.  Next,  observe  that 

if  yi  =  1  in  some  solution  then  Ax*  >  byi  —  b,  thus  ex'  >  z\.  Let  yj 

be  a  feasible  solution  to  P2,  then 

X)  (c*c)x‘‘  ^  XI  m** “  ]C  e*(cz‘)K 

1  <*<«  1<»<» 

—  —  »t-i  —  - 

>H  J]  CiVi  >  Zizi  =  (zi)2. 

1  <»<n 


Let  y*  be  the  optimal  solution  to  Pi,  then  we  claim  that  given  by 


y  =  V 


x*  _  /  y*  if  y*  =  i, 

\  0,  otherwise, 

is  an  optimal  solution  to  P2  since, 


53  (c,c)z‘  =  53  c<(«‘)y< 
1<<<» 

= 53  c*  Ft 

i 

=  (xi)2 
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thus  z3  =  [zi)2.  This  proves  the  first  part.  Again  let  ({**}?_ lt  y)  be  any  feasible 
solution  to  Pa  and  let  2  be  the  vector  that  minimizes 

min  <  cy,  min  ex *  > 

l  i<«<«  J 


so  that  el  <  ex*  and  c2  <  cy. 

v2  —  53  «*■(«*)  ^  13  c*(cz‘)w 

l^*<n  1<*S» 

>  £  *(«*)*  >  (e*)  X)  e<V<  ^  (C*)2 
!<»<»  1<‘<*» 


which  implies  that 

el  <  (t»a)^- 

This  proves  the  lemma  for  k  =  2. 

We  now  go  by  induction.  Assume  we  have  the  problem 


z*  =  minimum  dx 

Dx  >  h  (Pk) 

*;  =  0,1 

The  P*+ 1  problem  is 

zjt+i  =  minimum  53  (c,- d)z* 

i<»<» 

Dxi  —  hy,  >  0  (P*+i) 

Ay  >  6 

x)  =  0, 1  yi  =  0, 1 

Problem  Pu+i  i»  of  the  proper  form  if  Pi  and  P*  are.  Let  y)  be  a 

solution  to  Pfc^-i. 


53  M**  ^  51  >  13  c<(*k)V< 

1 


>  zk  53  cw  ^  ***»  =  (*ok*i  =  (*i)*+l- 
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Let  y*  be  the  optimal  solution  to  Pi,  and  we  claim  as  before  that  ({**}*_ x)  V 
given  by 


(  “optimal  solution  to  P**,  if  p, •  =  1, 

1 0,  otherwise, 


is  optimal  to  Pjt+i. 


2k+i  =  J2  c*'(rfz‘)  =  5Z  aidz^yi  =  23  ei(z*)yi 

l<«<n  1S*<»  1<*<w 

=  xk  23  ciVi  =  ZkZi  =  fa)*** =  (*i)*+1- 
1  <*<» 


Now  let  ({z‘}*_x)  yj  be  any  feasible  solution  with  value  t>*+i.  For  each  t  let  u* 

be  the  best  solution  obtained  to  the  Pi  problem  from  the  x%  vector  as  before. 
Let  u  be  the  best  among  the  «*’s  and  y  so  that  by  induction  we  have 


c«‘  <  (dr’)17* 

and 

cu  <  cu*  cu  <  cy. 

Finally, 

w*+i  =  21  c*(dz‘)  ^  H  «<(**)*<  ^  5Z  ct(cu‘)*yt- 
1<»<«  l<i<n  l<<<n 

>  (cu)*  21  c»v»'  —  (cu)*cu  =  (c«)fc+1 

1  <  i<  n 


cu  <  (t-»+i),/(*+,).  , 

We  can  now  prove  the  theorem.  Assume  we  have  a  problem  Pi  and  an 
cZ 

algorithm  for  which  —  <  r.  Let  e  >  0  fixed.  Pick  k  large  enough  so  that 

cz* 

r1/*  <  1  -j-  e.  Note  that  k  is  fixed  since  it  depends  only  on  the  fixed  e.  Now 
construct  problem  P*  from  Pi  as  above.  Apply  the  algorithm  to  P*  and  obtain 
a  solution  (with  value  v*).  From  this  we  construct  a  solution  to  Pi  (with  value 
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«i).  The  lemma  above  says  that  (zi)k  =  Zk  and  (t>i)*  <  w*  thus  we  have 


_  («i)* 
(*i)k 


<  —  <  r 
Zk 


which  implies 

—  <  r1/*  <  1  +  c. 

Zl 

If  the  original  problem  Pi  had  n  variables  and  m  constrants  then  the  problem 
P*  would  have  n  n*  var^bl®8  and  m  ]£<,= o  n*  constraints.  If  our  algorithm 
ran  in  0(p(m,  n))  time  and  gave  at  worst  an  error  of  r,  applying  it  to  P^  above 
would  give  a  solution  procedure  that  ran  in  0(p(mn\  nk+l))  time  and  gave  an 
error  of  r1/*.  Thus  either  there  is  no  polynomial  approximation  scheme  that  has 
a  fixed  error  bound,  or  there  is  a  polynomial  approximation  scheme  that  can  be 
adjusted  to  give  an  error  bound  of  1  +  e  for  arbitrary  c  >  0.  | 

We  turn  now  to  an  even  simpler  problem,  the  set  covering  problem  with  unit 
costs.  Presumably  this  is  a  more  tractable  problem.  From  Theorem  2.1  we  know 
the  error  is  bounded  by  H{k)  where  the  maximum  number  of  ones  in  any  column 
is  A.  An  example  where  the  error  is  exactly  H(k)  from  Johnson[3]  is  given  below 
for  the  case  k  —  3.  Here  c  =  (1, . . . ,  1)  and  6  =  (1, . . . ,  1). 


I 


(0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) 
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x*  =  (1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) 

It  might  appear  that  all  that  is  needed  to  reduce  the  error  substantially  is  to  cnd 
a  proper  tie-breaking  rule.  That  is,  some  way  to  decide  among  the  columns  that 
have  the  longest  lengths.  Unfortunately,  we  have 

Theorem  5.3.  Applying  the  greedy  heuristic  to  the  set  covering  problem  with 
unit  costs,  no  tie- breaking  rule,  i.e.  a  rule  that  chooses  between  columns  of  the 
same  length,  can  guarantee  an  error  less  than  H{ k)  — 

Proof.  We  construct  an  example  where  the  column  choice  is  forced  for  the  greedy 
algorithm  and  any  tie-breaking  rule  would  not  change  the  column  choices.  The 
matrix  A  is  constructed  for  the  case  of  k  =  4  (see  next  page);  the  extension 
to  arbitrary  k  should  be  clear.  First  construct  a  matrix  D  with  k  columns  and 
k{k  —  1)  rows  where  each  column  has  (k  —  1)  l’s. 

\ 

1 

1 


1 

1 


1  J 

Create  a  block  diagonal  matrix  with  the  matrix  D  on  the  diagonal.  The  columns 
of  this  will  make  up  the  optimal  solution.  The  greedy  heuristic  covers  the  first  k 
rows  of  each  copy  of  D  with  columns  of  length  k,  the  next  k  rows  with  columns  of 
length  k  —  1,  ...  the  next  to  last  k  rows  with  columns  of  length  3  and  the  last  k 
rows  with  columns  of  length  1.  The  entire  case  k  =  4  is  below.  At  each  iteration 
the  greedy  algorithm  picks  a  column  of  maximum  length  and  no  tie-breaking 
rule  would  ever  be  used  to  any  advantage.  | 


n 
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the  entries  in  A,  b,  and  c  are  all  non-negative.  The  first  heuristic  is  for  the  case 
where  the  entries  in  A  and  6  are  integral,  the  second  only  assumes  the  rows  are 
scaled  so  that  the  smallest  nonzero  entry  is  at  least  1.  In  both  cases  we  compare 
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